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DETAILED ACTION 

1. This action is responsive to communication: Amendment filed 08/02/2005, with 
acknowledgement of original filing date of 03/07/2004, which was benefited from foreign 
priority No. 12184/2001 filed 03/09/2001. 

2. Claims 1-20 are currently pending in this application. Claims 1, 13 and 17 are 
independent claims. 

Response to Argument 

3 . Applicant's arguments with respect to claims 1-20 have been considered but are moot in 
view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. I0l reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 1-20 are rejected under 35 U S.C. 101 because the claimed invention is directed 
to non- statutory subject matter. The claims invention set forth non-functional descriptive 
material but fails to set forth physical structures or materials comprising of hardware or a 
combination of hardware and software within the technological arts (i.e. a computer) to produce 
a "useful, concrete and tangible" result. Claims 1-20, are interpreted as software per se, abstract 
ideas or mental construct and not tangibly embodied on a computer readable medium or 
hardware. 

Further, to expedite a complete examination of the instant application the claims rejected 
under 35 U.S.C. 101 (nonstatutory) above are further rejected as set forth below in anticipation 
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of applicant amending these claims to place them within the four statutory categories of 
invention. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-20 are rejected under 35 U.S.C. 103(a) as being unpatentable by Gibbon et al. 
US006714909B1 - filed 1 1/21/2000 (hereinafter Gibbon '909), in view of Nelson et al. 
US006243713B1 - filed 08/24/1998 (hereinafter Nelson £ 713). 

In regard to independent claim 1, extracting a plurality of text areas from a video 
stream (Gibbon '909 at col. 2, lines 1-30, discloses ability to segment multimedia data, such as 
news broadcasts, into retrievable units that are directly related to what users perceive as 
meaningful, such as separating a multimedia data stream into audio, visual and text components, 
segmenting the audio, visual and text components based on semantic differences), 

calculating importance measures according to weights for each of the extracted text 
areas (Gibbon '909 at col. 8, line 45 through col. 14, line 40, also see Fig. 13-Fig. 17, discloses a 
mechanism to recover the semantic structure of the data for creating appropriate descriptions of 
the extracted multimedia content, such as: 

(i) To present the semantic structure to the users, 

(ii) To represent the particular semantics based on the content of the news story, 
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(iii) To form the representation for news summary of the day, 
Wherein textual and another is combination of text with visual is to automatically 
construct the representation in a form that is most relevant to the content of the underlying story 
according to their importance computed as weighted frequency (see Gibbon c 909 at col. 8, line 
10 through col. 9 through col. 12, line 5 for detail of the calculation steps and formula of the 
importance computed as weighted frequency) Examiner read the above in the broadest 
reasonable interpretation to the claim limitation, wherein calculating importance measures would 
have been an obvious variant of computed as weighted frequency. 

Gibbon c 909 does not explicitly teach, synthesizing the number of text areas into a 
synthetic key frame, however (Nelson '713 at col. 6, lines 5-50, discloses compound documents 
which are separated into constituent multimedia components of different data types, such as text, 
images, video, audio/voice, and other data types with portion thereof. Preferably these various 
multimedia components are combined with one or more query operators includes both text and . 
image components, and a number of query operators defining both logical relationships and 
proximity relationships between the multimedia components) Examiner read the above in the 
broadest reasonable interpretation to the claim limitation, wherein synthetic key frame would 
have been an obvious variant of separated into constituent multimedia components of different 
data types and combined with one or more query operators includes both text and image 
components), 

Selecting a number of text areas to be synthesized based upon the importance 
measures in the order of higher importance, however (Nelson '713 at col. 6, lines 5-50, 
however (Nelson '713 at col. 6, lines 5-50, discloses compound documents which are separated 
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into constituent multimedia components of different data types, such as text, images, video, 
audio/voice, and other data types with portion thereof Preferably these various multimedia 
components are combined with one or more query operators includes both text and image 
components, and a number of query operators defining both logical relationships and proximity 
relationships between the multimedia components) Examiner read the above in the broadest 
reasonable interpretation to the claim limitation, wherein synthetic key frame and based upon the 
importance measures would have been an obvious variant of separated into constituent 
multimedia components of different data types and combined with one or more query operators 
includes both text and image components and logical relationships and proximity relationships 
between the multimedia components, to a person of ordinary skill in the art at the time the 
invention was made . 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Gibbon '909, discloses a method of extracting a plurality 
of text areas from a video stream and calculating importance measures according to weights for 
each of the extracted text areas, to include a means of synthesizing the text areas to be 
synthesized into the key frame and selecting the number of text areas to be synthesized based 
upon the importance measures in the order of higher importance of Nelson '713. One of the 
ordinary skills in the art would have been motivated to perform such a modification to provide 
a desirable system that retrieves compound documents in response to queries that include 
various multimedia elements in a structured form, including text, image features, audio, or 
video (as taught by Nelson '713 at col. 2, lines 10-20). 
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In regard to independent claims 13 and 17, incorporate substantially similar subject 
matter as cited in claim 1 above, and are similarly rejected along the same rationale. 

In regard to dependent claim 2, wherein the text areas are extracted according to 
certain intervals of the video stream (Gibbon '909 at col. 14, lines 15-35, discloses the steps 
wherein during playback, audio is synchronized with video. Either key frames or the original 
video stream is played back. The text scrolls up with time. In the black box at the bottom, the 
timing with respect to the starting point of the program is given. . .). 

In regard to dependent claim 3, wherein a synthetic key frame is generated for each 
of the certain intervals of the video stream (Gibbon '909 at col. 13, lines 15-35 discloses the 
steps during playback, audio is synchronized with video. Either key frames or the original video 
stream is played back. The text scrolls up with time. In the black box at the bottom, the timing 
with respect to the starting point of the program is given. . .). 

In regard to dependent claim 4, wherein the certain intervals of the video stream are 
discriminated by scenes as logical units of a video (Gibbon '909 at col. 10, line 15 through col. 
1 1, line 40, also see Fig. 12, discloses a stream of detected audio events where A stands for 
anchor's speech, D stands for detailed reporting (from non-anchor people), and C stands for 
commercials. The center timeline in FIG. 12 shows the segments of text obtained from the text 
event segmentation (unit 405) using marker A where the duration of each segment does not 
include commercials) Examiner read the above in the broadest reasonable interpretation to the 
claim limitation, wherein certain intervals and logical units would have been an obvious variant 
of timeline the segments of text obtained from the text event segmentation using markers A, C an 
D, to a person of ordinary skill in the art at the time the invention was made . 
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In regard to dependent claim 5, wherein the certain intervals of the video stream are 
discriminated by shots as physical units of a video (Gibbon '909 at col. 10, line 15 through 
col. 1 1, line 40, also see Fig. 12, discloses a stream of detected audio events where A stands for 
anchor's speech, D stands for detailed reporting (from non-anchor people), and C stands for 
commercials. The center timeline in FIG. 12 shows the segments of text obtained from the text 
event segmentation (unit 405) using marker A where the duration of each segment does not 
include commercials) Examiner read the above in the broadest reasonable interpretation to the 
claim limitation, wherein certain intervals and logical units would have been an obvious variant 
of timeline the segments of text obtained from the text event segmentation using markers A, C an 
D, to a person of ordinary skill in the art at the time the invention was made . 

In regard to dependent claim 6, incorporate substantially similar subject matter as cited 
in claim 1 above and in further view of the following, and is similarly rejected along the same 
rationale, ...a display duration time of a text (Gibbon '909 at col. 10, line 15 through col 1 1, 
line 40, also see Fig. 12, discloses a center timeline wherein the segments of text obtained from 
the text event segmentation (unit 405) using marker A, C an D. . .). 

In regard to dependent claim 7, wherein the mean text size in the text area is 
determined by using a density and size of a histogram for the text area (Gibbon '909 at col. 
13, line 60 through col. 14, line 5, also see Fig. 15, discloses a keyword histogram is first 
constructed a fixed number of key frames within the boundary are chosen so that they (1) are not 
within anchor speech segments and (2) yield maximum covered area with respect to the 
keywords histogram. The peak points marked on the histogram in FIG. 15 indicate the positions 
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of the chosen frames and the shaded area underneath them defines the total area coverage on the 
histogram by the chosen key frames. . .). 

In regard to dependent claim 8, wherein the display duration time of the text is 
determined by considering whether a previously extracted text area is identical to a 
currently extracted text area (Gibbon '909 at col. 10, lines 50-65, discloses the block of text 
available at this point, the task is to determine how these blocks of text can be merged to form 
semantically coherent content based on appropriate criteria. Since news introductions are to 
provide a brief and succinct message about the story, they naturally have a much shorter duration 
than the detailed news reports. Based on this observation, in step 5060, a headline story 
segmentation unit 440 initially classifies each block of text as a story candidate or an 
introduction candidate based on duration. Also Gibbon '909 at col. 12, lines 15-25 discloses 
blocks formed in this way not only contain enough information for similarity comparison but 
also have natural breaks of chains of repeated words if true boundaries are present. . .) Examiner 
read the above in the broadest reasonable interpretation to the claim limitation, wherein 
determined by considering whether a previously extracted text area is identical to a currently 
extracted text area would have been an obvious variant of t block of text available at this point 
and contain enough information for similarity comparison but also have natural breaks of chains 
of repeated words if true boundaries are present , to a person of ordinary skill in the art at the 
time the invention was made . 

In regard to dependent claim 9, wherein the weight increases as the size of the text 
area, the mean text size of the text area or the display duration time of the text increases, 
(Gibbon '909 at col. 13, lines 30-50, also see Fig. 14, discloses a window that plays back 
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streaming content to a user. It is triggered when users click on a particular item. In this playback 
window, the upper portion shows the video and the lower portion the text synchronized with the 
video. During playback, audio is synchronized with video. Either key frames or the original 
video stream is played back. The text scrolls up with time. In the black box at the bottom, the 
timing with respect to the starting point of the program is given. . . keywords are chosen in step 
5080 above, from the story according to their importance computed as weighted frequency). 

In regard to dependent claims 10-12, 15-16 and 18 incorporate substantially similar 
subject matter as cited in claim 1 above, and are similarly rejected along the same rationale. 
Examiner read the above in the broadest reasonable interpretation to the claim limitation, 
wherein the certain rule is addition of values obtained by multiplying the weight determining 
factors with the corresponding weights and wherein the weight determining factors would have 
been an obvious variant of calculating importance measures according to weights, to a person of 
ordinary skill in the art at the time the invention was made. 

In regard to dependent claim 14, incorporate substantially similar subject matter as cited 
in claim 6 above, and is similarly rejected under the same rationale. 

In regard to dependent claim 19, incorporate substantially similar subject matter as cited 
in claim 7 above, and is similarly rejected under the same rationale. 

In regard to dependent claim 20, incorporate substantially similar subject matter as cited 
in claim 8 above, and is similarly rejected under the same rationale. 
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Conclusion 



8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Quoc A. Tran whose telephone number is (571) 272-4103. 
The examiner can normally be reached on Monday through Friday from 9 AM to 5 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Herndon R. Heather can be reached on (571) -272-4136. The fax phone number 
for the organization where this application or proceeding is assigned is (571)-273-8300 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access 
to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). 

Quoc A, Tran 
Patent Examiner 
Technology Center 21 76 
October 5, 2005 




